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© Voice password-controlled computer security system. 



© A voice password-controlled security system is 
disclosed. After the normal password procedure is 
successfully completed, the computer (12) institutes 
a voice call to a telephone (13) associated with the 
user. The computer queries the user to repeat a 
randomly selected series of digits or a phrase con- 
sisting of a group of words. The computer then 
compares the voice information received over the 
voice line with pre-stored voice information asso- 

FIG. 1 



ciated with the purported user. The computer affords 
access to its resources if and only if a voice match 
occurs. Due to the high reliability of the voice rec- 
ognition system, intruders, even those with password 
information and who mimic the user's voice, will 
usually be precluded access to the computer. Sig- 
nificantly, the system is effective yet places no addi- 
tional burdens on the user, such as memorizing 
additional codes or carrying encryption devices. 
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Technical Field 

This invention relates to computer system se- 
curity and, in particular, to password-controlled 
computer systems. 

Background of the Invention 

In many industries, computers are relied upon 
to store and process highly sensitive information. 
The banking industry uses computers to control 
fund transfers and the dispensing of cash at auto- 
mated teller machines. The telecommunication in- 
dustry uses computers to establish call connec- 
tions around the globe as well as to process asso- 
ciated billing information. Other industries control 
inventory, generate bills, and control medical in- 
struments all through the use of computers. Natu- 
rally, such computers are inviting targets to inter- 
lopers - both the professional variety who seeks 
access for profit and the so-called "hacker" who 
purportedly seeks access for fun and glory. 

Typically, security in computer systems has 
been instituted through the use of preassigned 
passwords. In response to a computer query, a 
user who supplies the correct password is given 
access to the computer system, while one who fails 
to provide the the password is not afforded such 
access. The use of passwords is generally effec- 
tive, yet is often easily defeated mainly due to 
human failings. Users, to avoid forgetting their 
password, sometimes write it down in a convenient 
place - like on the back of their automatic teller 
card or on the terminal-itself. Moreover they often 
select passwords that are equally obvious: their 
first or last name, their social security number, their 
children's name, their home address or their tele- 
phone number. Moreover, certain operating sys- 
tems provide standard passwords (e.g., "install") 
which should be changed by the System Admin- 
istrator during the initial installation of the computer 
system. However, many Systems Administrators 
fail to change these standard passwords leaving 
their systems particularly vulnerable. In short, the 
security of password-controlled systems is often 
breached because interlopers find passwords or 
are able to quickly guess passwords with a few 
intelligent choices. To help overcome these prob- 
lems, security in password controlled systems have 
been augmented by a user-controlled calculator- 
like device, adapted to execute a secret encryption 
algorithm. After the computer receives a valid pass- 
word, it sends a number to the terminal of the user. 
The user then manually enters this number into the 
calculator. The calculator then automatically ex- 
ecutes the secret algorithm to generate an output 
number. The user enters the output number and 
the computer compares this number with a similar 



number it generates internally using the same al- 
gorithm. A match indicates that the user possesses 
this unique encryption calculator and access to the 
computer is provided. This system is highly effec- 

5 five, except obviously where the intruder has both 
the password and the calculator. 

The basic problem with these security sys- 
tems, however, is that they allow access by a user 
without ascertaining his or her true identity - i.e., 

10 ' without establishing some personal uniqueness, 
such as through fingerprints, DNA criteria or, as in 
the present invention, voice characteristics. 
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Summary of the Invention 



i; The foregoing problem is solved by a voice 
password-controlled security system. After the nor- 
mal password procedure is successfully complet- 
ed, the computer institutes a voice call to a tele- 
20 phone associated with the user. The computer que- 
ries the user to repeat a randomly selected series 
of digits or a phrase consisting of a group of words. 
The computer then compares the voice information 
received over the voice line with pre-stored voice 
25 information associated with the purported user. The 
computer affords access to its resources if and 
only if a voice match occurs. Due to the high 
reliability of the voice recognition system, intruders, 
even those with password information and who 
30 mimic the user's voice, will usually be precluded 
access to the computer. Significantly, the system is 
effective yet places no additional burdens on the 
user, such as memorizing additional codes or car- 
rying encryption devices.- 

35 In accordance with an embodiment of the in- 

vention, a voice password-controlled computer sys- 
tem first queries a user for a -login" and then a 
"password" at a terminal. If the user enters a login 
and password at the terminal corresponding to 
4o\ those of a user entitled to access the computer, the 
computer then requests the user at the terminal to 
provide a telephone number identifying a voice 
telephone in proximity to the user. Alternatively, in 
more secure environments, a voice telephone num- 
45 ber may be preassigned for that user to ensure 
that the user is at a particular telephone before 
computer access in afforded. The computer then 
institutes a "voice" call to the telephone identified 
by the received or preassigned telephone number 
so and then orally requests the user to repeat a series 
of randomly selected digits, such as "one" "seven" 
"five" "one". Using a voice recognition technique, 
the computer matches received voice information 
with pre-stored voice information for the user and 
55 generates a confidence recognition factor indicating 
how closely the received voice matches the stored 
voice of the user. If the factor exceeds a preset 
threshold, the user is afforded access to the com- 



BNSDOCID: <EP_ 



EP 0 444 351 A2 



puter. The voice line is dropped since it is no 
longer needed. 

In accordance with a feature of the invention, 
the digits or words are chosen at random to pre- 
vent an intruder from merely tape recording a prior 5 
voice session and playing it back over the voice 
line to deceive the computer. 

Brief Description of the Drawings 

10 

FIG. 1 illustrates in block diagram form, a voice 
password-controlled computer security system em- 
bodying the principles of the instant invention. 

FIG. 2 sets forth in greater detail the voice 
apparatus and programmed processor shown in 75 
Fig. 1. 

FIGS. 3 and 4 illustrate flow charts of the 
operation of the system shown in FIG. 1 in accor- 
dance with the principles of the invention. 

FIG. 5 sets forth an illustrative section of the 20 
data base stored in the memory of the program 
processor shown in FIG. 2. 

Detailed Description 

25 

In accordance with the illustrative embodiment 
of this invention, user terminal 11 in FIG. 1— which 
may be a personal computer, a terminal or other 
peripheral device with a keyboard and a display — 
establishes a data connection to programmed pro- 30 
cessor 12 In the normal manner. This data connec- 
tion may, as shown in FIG. 1, be over a private 
line, a local area network, a wide area network, or 
even over the public switched network using a 
modem. Indeed, the user terminal may even be 35 
directly connected locally to the processor. Pro- 
grammed processor 12 may be any type of gen- 
eral purpose computer comprising memory, a cen- 
tral processing unit and ports through which remote 
terminals may establish data connections. In this 40 
illustrative embodiment, processor 12 is a 3B2- 
1 000 computer manufactured by AT&T and running 
the UNIXTM operating system. 

Processor 12, in response to the initiation of a 
bidirectional data connection by terminal 11, pro- 45 
vides a "LOGIN" prompt to the user at the termi- 
nal. The user then enters his or her login which 
identifies the user, and processor 12 then prompts 
the user to enter a "PASSWORD". If the login and 
password information input by the user, match pre- 50 
stored login and password information maintained 
in memory by processor 12, the processor then 
independently attempts to establish a voice con- 
nection to user telephone 13— which ideally is in 
close physical proximity to user terminal 11. This 55 
voice connection can be established either to a 
pre-assigned telephone number associated with the 
user or to a telephone number input by the user at 



terminal 11 in response to a query by processor 
12. This bidirectional voice connection is estab- 
lished in the normal manner through telephone 
central office switch 15 to user telephone 13. Al- 
though the voice connection is shown as a distinct 
physical path from the data connection, it need not 
be physically separate. Using present ISDN tech- 
nology, the voice connection and the data connec- 
tion can be over the same physical medium. More- 
over, the voice connection could just as readily be 
"hard-wired" to a locally associated processor 12. 
After establishment of the voice connection, pro- 
cessor 12 generates a 4-digit random number (e.g., 
5772) and controls voice apparatus 14 to request 
the user to repeat the 4-digit number into the user 
telephone 13 (e.g., "After the tone, please speak 
the following numbers in sequence: five, seven, 
seven, two"). Alternatively, processor 12 could que- 
ry the user by providing this same message in text 
form to the user via terminal 11. Processor 12 then 
compares the speech embodied in the received 4- 
digit number with the user's stored reference 
speech for the 4-digit number. A confidence, rec- 
ognition factor indicating the closeness of the 
match of the received speech patterns with the 
stored reference patterns is assigned to the re- 
ceived speech. This factor is then compared with a 
pre-determined threshold value established to iden- 
tify valid "voice passwords." If the confidence rec- 
ognition factor is greater than the threshold value, 
the user is afforded access to processor 12 and 
may request the transfer of funds or access to a 
proprietary data base or other functions provided 
by the processor. The voice connection to tele- 
phone 13 is dropped since it is no longer needed. 

The operation of the illustrative embodiment 
shown in FIG. 1 will now be described in greater 
detail with reference to the flow diagram of FIGS. 3 
and 4 and the illustrative data shown in FIG. 5. We 
will assume that the user associated with login 
AFC4 in FIG. 5 is at user terminal 11 in FIG. 1 and 
wishes to access certain restricted programs in 
programmed processor 12. The user associated 
with login AFC4 initiates a data connection to pro- 
cessor 12 in the manner described above. Proces- 
sor 12 then prompts the user first for a login and 
then for. a password, step 31 in FIG. 3. If incorrect 
login or password information is entered by the 
user, processor 12 will terminate the session by 
dropping the data connection, step 33. However, if 
the user enters login W AFC4", and password 
"BANANA11", processor 12, upon determining that 
a match occurs since the same login and password 
are stored in table 61 of FIG. 5, will afford the user 
access to pre-selected, non-critical computer re- 
sources, step 32 in FIG. 3. Such resources may be 
certain non-proprietary programs, or certain de- 
vices such as printers. Alternatively, such re- 
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sources may, in fact, constitute no resources at all 
in systems containing highly sensitive information. 
Thus, the system administrator for processor 12 
should make a determination whether to protect the 
security of the whole computer system by requiring 5 
voice identification for all users; whether to only 
require such voice identification when a user at- 
tempts to access certain sensitive programs, such 
as a data base, or external links to other networked 
computers; or whether to only require voice iden- 70 
tification for certain users and not others. One such 
illustrative determination by the system administra- 
tor is reflected in the column entitled "critical ac- 
cess" in table 61 of FIG. 5. Users AFC4 and JHL 
are entitled to access critical resources, such as a 75 
highly confidential Informix data base, which are 
protected by voice identification as hereinafter ex- 
plained, while SLL is not entitled to any access 
whatsoever to such critical resources. 

If user AFC4 seeks access to the Informix data 20 
base by entering the code "isql\ processor 12 will 
first ascertain by reference to table 61 in FIG. 5, 
whether such access is allowed. Here, user AFC4 
is entitled to such critical access. User SLL, on the 
other hand, is not entitled to access and would be 25 
so apprised at step 36 in FIG. 3 by the words 
"ACCESS DENIED". 

With respect to the "isql" access request by 
user AFC4, processor 12 next ascertains whether 
or not a voice telephone number has been pre- 3a 
assigned, step 34, by accessing the information in 
the column entitled "preassigned voice number" in 
table 61. Pre- assigning a voice telephone number 
associated with a particular user (e.g., user "JHL") 
is highly effective in precluding access from any 35 
other physical location. However, if the System 
Administrator did not wish to so limit remote ac- 
cess for a particular user such as for AFC4, no pre- 
assigned voice number would be stored in the 
password storage table in FIG. 5. Processor 12 40 
therefore queries user terminal 11 for a telephone 
number of a voice telephone located near the ter- 
minal, step 35, FIG. 3. 

Processor 1 2, upon receiving a voice telephone 
number entered from user terminal 11, uses a look- 45 
up table to ascertain the validity of the entered 
number. This validity check can be used to ensure 
that telephone cails are only completed to certain 
telephone numbers, telephone exchanges, or area 
codes - another security feature. If the received so 
number is valid, processor 12 initiates a voice 
telephone call to user telephone 13 via voice ap- 
paratus 14, shown in greater detail in FIG. 2. 

Voice apparatus 14 comprises voice response 
unit 22 and digit recognizer 23- both intercon- 55 
nected by bus 21 and controlled by processor 12. 
To provide speech output, processor 12 provides 
stored text in a digital format to voice response unit 



22, which using a digital to analog converter gen- 
erates speech. Typically, phrases such as "access 
granted", "nine", "after the tone ...", are stored in 
data storage device 24 in a digital format as shown 
in table 63 in FIG. 5. Processor 12 generates and 
provides speech to a user by providing such digital 
information over bus 21 to voice response unit 22, 
and controlling unit 22 to output the speech over 
the voice connection shown in FIG. 2. Digits spok- 
en by a user are detected by digit recognizer 23, 
which includes an analog to digital convertor for 
converting received spoken digits into a digital for- 
mat and for conveying the digital information, upon 
request to processor 12 via bus 21. 

If the call to user telephone 13 is answered and 
a voice connection established (step 37, FIG. 3), 
processor 12 randomly generates a 4-digit number 
(step 41 , FIG. 4) and retrieves from data storage 24 
the four corresponding stored reference speech 
digits for the user. If the random number were 
"9102", processor 12 would retrieve from table 62 
in FIG. 6 for user AFC4, the digital information 
corresponding to "NINE" "ONE" ZERO" and 
"TWO". Processor 12 also retrieves binary infor- 
mation from table 63 representing the phrase 
"After the tone, please speak the following number 
in sequence" and routes it over bus 21 to voice 
response unit 22, which converts the binary in- 
formation to speech. The binary information repre- 
senting each of the four digits of the randomly 
chosen number "9102" is also retrieved from table 
63 and conveyed in sequence over bus 21 to unit 
22 where it is also converted to speech. Thus user 
AFC4, (step 42, FIG. 4) is asked over the voice 
connection in FIG. 1 to repeat the digits "9102" 
into user telephone 13. User AFC4 then repeats the 
digits "9102" into telephone 13 for conveyance 
over the voice connection. Digit recognizer 23 in 
FIG. 2 digitizes the speech (step 43) and conveys 
the received speech to processor 12. Processor 12 
utilizing a well-known process, uses energy con- 
tours of the received speech to identify end points 
of each number, forms test patterns of frames 
consisting of cepstral co-efficients and speech en- 
ergy. A DTW procedure then calculates the dis- 
tance between the referenced speech pattern for 
user AFC4, which was retrieved previously, and the 
received test pattern and assigns a confidence 
recognition value thereto. If the calculated value is 
greater than a pre-set threshold (step 44), proces- 
sor 12 retrieves the phrase "access granted" from 
table 63 in FIG. 5, and transmits it via voice re- 
sponse unit 22 to user telephone 13 (step 45). The 
voice connection is then dropped and the user is 
afforded access to the requested critical computer 
resource - the Informix database. 

Although the instant embodiment has been de- 
scribed in terms of a randomly selected 4-digit 
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number, the voice-password security system works 
well with non-numeric phrases. The user reference 
voice storage 62 in FIG. 6 could, for example, 
comprise a series of commonly used phrases such 
as "the" "rain" "in" "Spain" "falls" "mainly" "on" 5 
"the" "plain" and "now" "is" "the" "time" "for" 
"all" "good" "men" "to" "come" "to" "the" "aid" 
"of "their" "party". With this vocabulary, the com- 
puter could query a user to repeat one of the 
following phrases: "The time for Spain is now" or io 
"The men come to the party in Spain" or "rain, 
men, party." 

It is obvious from the foregoing that the secu- 
rity of a computer system may be substantially 
enhanced by the described voice password sys- 75 
tern. While the instant invention has been disclosed 
with respect to a general purpose computer, it 
should be understood that such an embodiment is 
intended to be illustrative of the principles of the 
invention and that numerous other arrangements, 20 
such as a computer system dedicated to a specific 
use (e.g., automated teller machine functions), may 
be devised by those skilled in the art without 
departing from the spirit and scope of the inven- 
tion. 25 

Claims 

1. In a computer system in which a user at a 
terminal attempts to gain access to a computer 30 
storing preassigned password information and 
voice information associated with the user, the 
combination comprising 

means responsive to entry by the user of 
said preassigned password information at said 35 
terminal for initiating a voice connection to a 
telephone associated with the user, 

means for querying the user over said 
voice connection to speak a series of words 
into said telephone, and ao 

means for comparing words spoken by the 
user over said voice connection with said 
stored voice information and operable, upon a 
match thereof, to afford access to said com- 
puter by said terminal. 45 

2. The combination set forth in claim 1 where 
said querying means comprises means for ran- 
domly selecting said series of words. 

50 

3. In a computer system in which a user at a 
video display unit attempts to gain access to a 
computer connected thereto, the combination 
comprising 

means for storing a preassigned password 55 
identification associated with the user and a 
plurality of phrases spoken by the user, 

means responsive to an attempt by the 



user to gain access to the computer for 
querying the user for password identification, 

means for comparing password identifica- 
tion entered by the user at the video display 
unit with said preassigned password identifica- 
tion and operable, upon a match thereof, for 
querying the user to identify a telephone, 

means for initiating a telephone connection 
to the identified telephone, 

means for randomly selecting a series of 
phrases spoken by the user from said storing 
means, 

means for requesting the user over said 
telephone connection to repeat said randomly 
selected series of phrases, and 

means for comparing phrases spoken by 
the user over said telephone connection with 
said stored randomly selected series of 
phrases and operable, upon a match thereof, 
for granting access to said computer by the 
user via said video display unit. 

4. The method of preventing an unauthorized 
user at a terminal from accessing a computer 
system comprising 

comparing password information identify- 
ing a user and received from the terminal with 
password information for the user stored in the 
computer, 

only upon successfully matching the re- 
ceived and stored password information, estab- 
lishing a voice connection to a telephone asso- 
ciated with the user, 

querying the user over the voice connec- 
tion to speak a series of phrases, 

comparing the spoken phrases with stored 
voice information associated with the user, and 

only upon successfully matching the spok- 
en phrases with the stored voice information, 
granting the terminal access to the computer. 

5. The method of preventing an unauthorized 
user at a terminal connected to a computer 
from achieving access thereto, comprising the 
steps of: 

querying the user at the terminal to pro- 
vide password information, 

comparing received password information 
with stored password information to determine 
the validity of the received information, 

upon determining that the received pass- 
word is valid, querying the user to identify a 
telephone. 

establishing a voice connection to the tele- 
phone identified by the user, 

randomly selecting a series of phrases. 

querying the user over the voice connec- 
tion to repeat said series of phrases. 
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receiving phrases over the voice connec- 
tion from the user, 

comparing said received phrases with cor- 
responding stored phrases spoken by the user 
and maintained in said computer, and 

only upon successfully matching said re- 
ceived and stored phrases, granting the termi- 
nal access to the computer. 

6. A method of providing security in a computer 
system comprising the steps of 

storing spoken phrases of a user, 
subsequently prompting a user to enter 
voice phrases selected from said stored 
phrases and 

comparing phrases spoken by the user 
with the same stored phrases of the user and 
operable upon a match thereof for affording 
access to secured resources in the computer 
system. 

7. The method in accordance with claim 6 in 
which the voice phrases are randomly select- 
ed. 

8. The method of operating a computer system 
storing unique password and voice access 
commands comprising the steps of 

prompting a user first to enter the unique 
password and then to enter through speech 
selected of the access commands, 

comparing entered information first with 
the stored password and then with the stored 
selected voice access commands and oper- 
able upon a match of the entered information 
with both for affording access to resources in 
the computer system. 

9. A computer system adapted to store voice 
phrases comprising 

means for selecting a series of voice 
phrases associated with a user of the com- 
puter system and stored therein, 

means for prompting a user to verbally 
enter the selected series of phrases into the 
computer system, 

means for comparing the series of phrases 
verbally entered by the user with the selected 
stored series of phrases and operable upon a 
match thereof for enabling the user to access 
resources of the computer system. 

10. The computer system of claim 9 wherein said 
selecting means randomly selects said series 
of voice phrases from the stored phrased asso- 
ciated with the user of the computer system. 

11. The computer system of claim 9 wherein said 



selecting means uses an algorithm to select 
said series of voice phrases from the stored 
phrases associated with the user of the com- 
puter system. 
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FIG. 3 
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FIG. 5 
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© A voice password-controlled security system is 
disclosed. After the normal password procedure is 
successfully completed, the computer (12) institutes 
a voice call to a telephone (13) associated with the 
user. The computer queries the user to repeat a 
randomly selected series of digits or a phrase con- 
sisting of a group of words. The computer then 
compares the voice information received over the 
voice line with pre-stored voice information asso- 
ciated with the purported user. The computer affords 
access to its resources if and only if a voice match 
occurs. Due to the high reliability of the voice rec- 
ognition system, intruders, even those with password 
information and who mimic the user's voice, will 
usually be precluded access to the computer. Sig- 
nificantly, the system is effective yet places no addi- 
tional burdens on the user, such as memorizing 
additional codes or carrying encryption devices. 
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